[QA] Move integration tests to vizro #950

l0uden · 2025-01-13T16:17:52Z

Description

Here's the first iteration of tests moving from vizro-qa to vizro repo.
Here I moved (and refactored) part of the default dashboard, test helpers and tests for filters, pages and themes. Also created CI for running the tests
The aim of this PR is to find the way of organising this tests in the proper way, so I can move the rest of them after in the same manner

Notice

I acknowledge and agree that, by checking this box and clicking "Submit Pull Request":
- I submit this contribution under the Apache 2.0 license and represent that I am entitled to do so on behalf of myself, my employer, or relevant third parties, as applicable.
- I certify that (a) this contribution is my original creation and / or (b) to the extent it is not my original creation, I am authorized to submit this contribution on behalf of the original creator(s) or their licensees.
- I certify that the use of this contribution as authorized by the Apache 2.0 license does not violate the intellectual property rights of anyone else.
- I have not referenced individuals, products or companies in any commits, directly or indirectly.
- I have not added data or restricted code in any commits, directly or indirectly.

…_integration_tests_to_vizro

for more information, see https://pre-commit.ci

github-actions · 2025-01-13T16:19:11Z

View the example dashboards of the current commit live on PyCafe ☕ 🚀

Updated on: 2025-02-04 14:28:51 UTC
Commit: 7ebae43

Link: vizro-core/examples/dev/

Link: vizro-core/examples/scratch_dev

Link: vizro-core/examples/visual-vocabulary/

Link: vizro-ai/examples/dashboard_ui/

…om/mckinsey/vizro into qa/move_integration_tests_to_vizro

for more information, see https://pre-commit.ci

…om/mckinsey/vizro into qa/move_integration_tests_to_vizro

for more information, see https://pre-commit.ci

…om/mckinsey/vizro into qa/move_integration_tests_to_vizro

petar-qb

This looks great to me so far! 🚀 Amazing work, @l0uden! 🥇

That said, I think we should discuss a few key topics on a call rather than communicating everything through PR comments. Moving all the tests from the vizro-qa repo to the vizro repo is a significant change and potentially introduces some risks (like unoptimised folder structure...). It’s also a great opportunity to optimise while changes are still manageable and cheap because when the full migration is complete, making similar updates could become much more complex.

What do you think about this @antonymilne @Joseph-Perkins?
To reduce risks and streamline the process, I propose setting up the call about the Vizro tests architecture. Addressing these points now will save time and effort later, and it will help us to merge this PR faster (than communicating these things over the PR comments), allowing @l0uden to proceed with the test migration smoothly.

Some topics/questions I'd like to discuss:

Should we add a dedicated test hatch environment?
Can we improve the folder structure for better maintainability?
Should we combine vizro-core/tests/integrations with vizro-core/tests/e2e (and apply the same approach to vizro-ai)?

l0uden · 2025-01-17T12:52:50Z

This looks great to me so far! 🚀 Amazing work, @l0uden! 🥇

That said, I think we should discuss a few key topics on a call rather than communicating everything through PR comments. Moving all the tests from the vizro-qa repo to the vizro repo is a significant change and potentially introduces some risks (like unoptimised folder structure...). It’s also a great opportunity to optimise while changes are still manageable and cheap because when the full migration is complete, making similar updates could become much more complex.

What do you think about this @antonymilne @Joseph-Perkins? To reduce risks and streamline the process, I propose setting up the call about the Vizro tests architecture. Addressing these points now will save time and effort later, and it will help us to merge this PR faster (than communicating these things over the PR comments), allowing @l0uden to proceed with the test migration smoothly.

Some topics/questions I'd like to discuss:

Should we add a dedicated test hatch environment?

Can we improve the folder structure for better maintainability?

Should we combine vizro-core/tests/integrations with vizro-core/tests/e2e (and apply the same approach to vizro-ai)?

Thanks @petar-qb !

Completely agree with you about the call about this, let's discuss time on Monday, when @antonymilne will be available. And also it would be better if he could have time to take a look on what's done.

…_integration_tests_to_vizro

maxschulz-COL

⭐ Looks good to me 🚀

I have some suggestions, please only implement them if they seem useful, and they make sense!

vizro-core/tests/tests_utils/e2e/checkers.py

vizro-core/tests/e2e/vizro/dashboards/default/dashboard_pages.py

vizro-core/tests/e2e/vizro/dashboards/wait-for-it.sh

vizro-core/tests/tests_utils/e2e/custom_feature_helpers/custom_actions/custom_actions.py

vizro-core/tests/tests_utils/e2e/helpers.py

…_integration_tests_to_vizro

petar-qb · 2025-01-27T13:13:03Z

Considering all that @maxschulz-COL suggested and everything we aligned on during last week’s call, I’ve prepared the following folder structure proposal. This structure closely resembles what we currently have but introduces the following changes:

Renames e2e/vizro/dashboards/ -> e2e/vizro/component_utils/
Moves tests_utils/e2e/custom_feature_helpers/ under e2e/vizro/custom_components/
Extracts e2e/vizro/dashboards/default/dashboard.py to a higher-level directory: e2e/vizro/dashboards/dashboard.py
Breaks e2e/vizro/dashboards/default/dashboard_pages.py into multiple files and renames the parent folder from default/ to pages/
Defines a new structure of e2e/vizro/test_screenshots/
Move all 6 files from tests_utils/e2e/ into the new e2e/vizro/utils/
Renames helpers.py -> waiters.py

e2e/
├── component_library/
│   └── test_component_library.py
├── vizro/
│   ├── components_utils/  (1. Renames `e2e/vizro/dashboards/` -> `e2e/vizro/component_utils/`)
│   │   ├── assets/
│   │   │   └── ...
│   ├── custom_components/  (2. Moves `tests_utils/e2e/custom_feature_helpers/` under `e2e/vizro/custom_components/`)
│   │   ├── custom_actions/
│   │   │   └── ...
│   │   ├── custom_charts/
│   │   │   └── ...
│   │   ├── custom_components/
│   │   │   └── ...
│   ├── dashboard.py  (3. Extracts `e2e/vizro/dashboards/default/dashboard.py` to a higher-level directory: `e2e/vizro/dashboards/dashboard.py`)
│   ├── pages/  (4. Breaks `e2e/vizro/dashboards/default/dashboard_pages.py` into multiple files and renames the parent folder from `default/` to `pages/`)
│   │   ├── filter_page/
│   │   ├── parameter_page.py
│   │   └── ...
│   ├── test_dom_elements/
│   │   ├── conftest.py
│   │   ├── test_charts.py
│   │   ├── test_filters.py
│   │   ├── test_pages.py
│   │   └── ...
│   ├── test_screenshots/  (5. Defines a new structure of `e2e/vizro/test_screenshots/`)
│   │   ├── screenshot_images/
│   │   │   └── ... 
│   │   └── test_screenshots.py
│   ├── utils/  (6. Move all 6 files from `tests_utils/e2e/` into the new `e2e/vizro/utils/`)
│   │   ├── asserts.py
│   │   ├── checkers.py
│   │   ├── constants.py
│   │   ├── navigation.py
│   │   ├── paths.py
│   │   └── waiters.py  (7. Renames `helpers.py` -> `waiters.py`)
tests_utils/
...

@maxschulz-COL @l0uden @antonymilne what do you think about the seven changes I suggested?

maxschulz-COL · 2025-01-27T17:07:14Z

Considering all that @maxschulz-COL suggested and everything we aligned on during last week’s call, I’ve prepared the following folder structure proposal. This structure closely resembles what we currently have but introduces the following changes:

Renames e2e/vizro/dashboards/ -> e2e/vizro/component_utils/

Moves tests_utils/e2e/custom_feature_helpers/ under e2e/vizro/custom_components/

Extracts e2e/vizro/dashboards/default/dashboard.py to a higher-level directory: e2e/vizro/dashboards/dashboard.py

Breaks e2e/vizro/dashboards/default/dashboard_pages.py into multiple files and renames the parent folder from default/ to pages/

Defines a new structure of e2e/vizro/test_screenshots/

Move all 6 files from tests_utils/e2e/ into the new e2e/vizro/utils/

Renames helpers.py -> waiters.py
e2e/
├── component_library/
│   └── test_component_library.py
├── vizro/
│   ├── components_utils/  (1. Renames `e2e/vizro/dashboards/` -> `e2e/vizro/component_utils/`)
│   │   ├── assets/
│   │   │   └── ...
│   ├── custom_components/  (2. Moves `tests_utils/e2e/custom_feature_helpers/` under `e2e/vizro/custom_components/`)
│   │   ├── custom_actions/
│   │   │   └── ...
│   │   ├── custom_charts/
│   │   │   └── ...
│   │   ├── custom_components/
│   │   │   └── ...
│   ├── dashboard.py  (3. Extracts `e2e/vizro/dashboards/default/dashboard.py` to a higher-level directory: `e2e/vizro/dashboards/dashboard.py`)
│   ├── pages/  (4. Breaks `e2e/vizro/dashboards/default/dashboard_pages.py` into multiple files and renames the parent folder from `default/` to `pages/`)
│   │   ├── filter_page/
│   │   ├── parameter_page.py
│   │   └── ...
│   ├── test_dom_elements/
│   │   ├── conftest.py
│   │   ├── test_charts.py
│   │   ├── test_filters.py
│   │   ├── test_pages.py
│   │   └── ...
│   ├── test_screenshots/  (5. Defines a new structure of `e2e/vizro/test_screenshots/`)
│   │   ├── screenshot_images/
│   │   │   └── ... 
│   │   └── test_screenshots.py
│   ├── utils/  (6. Move all 6 files from `tests_utils/e2e/` into the new `e2e/vizro/utils/`)
│   │   ├── asserts.py
│   │   ├── checkers.py
│   │   ├── constants.py
│   │   ├── navigation.py
│   │   ├── paths.py
│   │   └── waiters.py  (7. Renames `helpers.py` -> `waiters.py`)
tests_utils/
...
@maxschulz-COL @l0uden @antonymilne what do you think about the seven changes I suggested?

That sounds good to me! Thanks @petar-qb! All perfect except maybe dashboard.py. Could it be dashboard_for_testing.py? (super minor though, feel free to also ignore)

…_integration_tests_to_vizro

l0uden · 2025-01-29T17:30:18Z

@maxschulz-COL and @petar-qb, thanks for the amazing comments!

I'll try to summarise here the major changes I made.

Switched all of the moved tests to dash testing methods. That helped to delete a lot of legacy methods and make code more readable.
Added comments to my methods for more understanding.
Split dashboard pages to different files. I left dashboard into the default folder, because there will be also yaml dashboard.
Moved custom_components closer to the dashboard
Created vizro app under tests_utils/e2e with file that used only by vizro tests. asserts will be used both by vizro and component_library tests.

petar-qb

This looks really good now. 🚀

…_integration_tests_to_vizro

l0uden and others added 3 commits January 13, 2025 17:16

moved tests for filters, pages and themes

007e19b

Merge branch 'main' of https://github.com/mckinsey/vizro into qa/move…

2e11cff

…_integration_tests_to_vizro

[pre-commit.ci] auto fixes from pre-commit.com hooks

40b382c

for more information, see https://pre-commit.ci

l0uden and others added 7 commits January 13, 2025 17:20

add gunicorn to hatch.toml

33141fe

Merge branch 'qa/move_integration_tests_to_vizro' of https://github.c…

87229fb

…om/mckinsey/vizro into qa/move_integration_tests_to_vizro

add pythonpath to start dashboard

7033db5

[pre-commit.ci] auto fixes from pre-commit.com hooks

10d33c5

for more information, see https://pre-commit.ci

changed port

db51b4f

Merge branch 'qa/move_integration_tests_to_vizro' of https://github.c…

c6466f1

…om/mckinsey/vizro into qa/move_integration_tests_to_vizro

changed port

f77c394

l0uden requested a review from petar-qb January 14, 2025 10:02

l0uden and others added 2 commits January 14, 2025 11:23

moved tests to separate folder

a2101e5

[pre-commit.ci] auto fixes from pre-commit.com hooks

db94441

for more information, see https://pre-commit.ci

l0uden marked this pull request as ready for review January 14, 2025 10:28

l0uden requested review from huong-li-nguyen, nadijagraca, Joseph-Perkins, antonymilne, maxschulz-COL and lingyielia as code owners January 14, 2025 10:28

l0uden added 2 commits January 14, 2025 14:44

linting fix

9a26fe1

Merge branch 'qa/move_integration_tests_to_vizro' of https://github.c…

43503fc

…om/mckinsey/vizro into qa/move_integration_tests_to_vizro

petar-qb reviewed Jan 17, 2025

View reviewed changes

l0uden added 2 commits January 23, 2025 16:51

reworked tests structure

b410746

Merge branch 'main' of https://github.com/mckinsey/vizro into qa/move…

d3b8ce9

…_integration_tests_to_vizro

maxschulz-COL approved these changes Jan 24, 2025

View reviewed changes

Merge branch 'main' of https://github.com/mckinsey/vizro into qa/move…

61a48de

…_integration_tests_to_vizro

l0uden added 9 commits January 29, 2025 13:30

dash testing method. folder structure changed

c2dd5e9

Merge branch 'main' of https://github.com/mckinsey/vizro into qa/move…

b770685

…_integration_tests_to_vizro

failed artifacts fix

43295cd

failed artifacts fix

71233ab

failed artifacts fix

af2faf5

failed artifacts fix

81897f0

failed artifacts fix

d80377c

failed artifacts fix

23f7e45

fix grey style

8ed5bff

petar-qb approved these changes Jan 31, 2025

View reviewed changes

l0uden added 5 commits February 3, 2025 12:54

Merge branch 'main' of https://github.com/mckinsey/vizro into qa/move…

e4007a6

…_integration_tests_to_vizro

custom component change to support pydantic v2

8e76053

Merge branch 'main' of https://github.com/mckinsey/vizro into qa/move…

351dbbd

…_integration_tests_to_vizro

Merge branch 'main' of https://github.com/mckinsey/vizro into qa/move…

02f98d9

…_integration_tests_to_vizro

Merge branch 'main' of https://github.com/mckinsey/vizro into qa/move…

2b36c14

…_integration_tests_to_vizro

antonymilne approved these changes Feb 4, 2025

View reviewed changes

l0uden and others added 4 commits February 4, 2025 15:17

Merge branch 'main' into qa/move_integration_tests_to_vizro

e8c483d

added reruns

e70ad1a

added reruns

9d7e830

added pytest-rerunfailures

7ebae43

l0uden merged commit 384c5e6 into main Feb 4, 2025
36 checks passed

l0uden deleted the qa/move_integration_tests_to_vizro branch February 4, 2025 14:31

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[QA] Move integration tests to vizro #950

[QA] Move integration tests to vizro #950

l0uden commented Jan 13, 2025 •

edited

Loading

github-actions bot commented Jan 13, 2025 •

edited

Loading

petar-qb left a comment

l0uden commented Jan 17, 2025

maxschulz-COL left a comment

petar-qb commented Jan 27, 2025

maxschulz-COL commented Jan 27, 2025

l0uden commented Jan 29, 2025

petar-qb left a comment

[QA] Move integration tests to vizro #950

[QA] Move integration tests to vizro #950

Conversation

l0uden commented Jan 13, 2025 • edited Loading

Description

Notice

github-actions bot commented Jan 13, 2025 • edited Loading

View the example dashboards of the current commit live on PyCafe ☕ 🚀

petar-qb left a comment

Choose a reason for hiding this comment

l0uden commented Jan 17, 2025

maxschulz-COL left a comment

Choose a reason for hiding this comment

petar-qb commented Jan 27, 2025

maxschulz-COL commented Jan 27, 2025

l0uden commented Jan 29, 2025

petar-qb left a comment

Choose a reason for hiding this comment

l0uden commented Jan 13, 2025 •

edited

Loading

github-actions bot commented Jan 13, 2025 •

edited

Loading